QVAC-21921 ggml-speech: ACE-Step Oobleck VAE custom ops (snake, col2im_1d) — CPU + Metal#43
Merged
Merged
Conversation
Add the two custom ggml ops ACE-Step needs, with CPU kernels: - GGML_OP_SNAKE (snake activation used by the Oobleck VAE) - GGML_OP_COL2IM_1D (1D col2im for transposed conv upsampling) Bump GGML_MAX_NAME 64 -> 128 (guarded) to fit ACE-Step GGUF tensor names, matching acestep.cpp's build override. Add op unit tests and a VAE decode/roundtrip test harness.
… COL2IM_1D) Add Metal GPU kernels for the two custom ops landed as CPU-only earlier, so the ACE-Step Oobleck VAE decode can run on GPU (Metal) instead of CPU: - kernel_snake_f32: per-channel snake activation y = x + sin^2(a*x) * inv_b - kernel_col2im_1d_f32: scatter-add col2im for the decoder's 1D transposed convs, flat grid-stride so occupancy stays high even when OC is tiny (final stereo transpose-conv) Wire both through the device pipeline cache, the op dispatch/encoder, and ggml_metal_device_supports_op (contiguous f32 in/out). ~4x faster VAE render on Apple Silicon (M1 Ultra) with output perceptually identical to the CPU path.
GustavoA1604
requested changes
Jul 21, 2026
- tests: replace POSIX mmap loader with portable fopen/fread so the VAE test targets build on Windows (ggml-speech is cross-platform) - ggml.h: document GGML_MAX_NAME 64->128 as a hard-rebuild ABI change (value kept) - drop QVAC-21921 ticket refs from code comments - remove "what" layout restatements in col2im_1d (ops.cpp + metal); keep the "why" Validated: clean build w/ tests; test-snake, test-col2im-1d, test-vae-graph pass.
GustavoA1604
approved these changes
Jul 22, 2026
ogad-tether
approved these changes
Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the two custom ggml ops the ACE-Step Oobleck VAE / DiT graphs rely on, so
the audiogen (ACE-Step music generation) engine can run on this fork. Lands
both the CPU reference implementations and the Metal GPU kernels, so the VAE
decode runs on Apple Silicon (Metal) as well as CPU.
Part of QVAC-21921 (native ACE-Step music generation). Consumed downstream by
the
audiogen-cppengine (qvac-ext-lib-whisper.cpp) via theggml-speechvcpkg port, which builds from this branch.
What's in here
New ops (
GGML_OP_SNAKE,GGML_OP_COL2IM_1D):ggml_snake: per-channel snake activationy = x + sin^2(a*x) * inv_b(Oobleck VAE activations).
ggml_col2im_1d: scatter-add col2im for the decoder's 1D transposed convs.op metadata.
CPU backend (commit 1): reference implementations for both ops.
Metal backend (commit 2):
kernel_snake_f32andkernel_col2im_1d_f32, wired through the devicepipeline cache, the op dispatch/encoder, and
ggml_metal_device_supports_op(contiguous f32 in/out).
(the final stereo transpose-conv).
Scope is additive: only the two new ops + their CPU/Metal kernels. No changes to
existing ops.